The approximate coordinate exchange algorithm for Bayesian optimal design of experiments

Optimal Bayesian experimental design typically involves maximising the expectation, with respect to the joint distribution of parameters and responses, of some appropriately chosen utility function. This objective function is usually not available in closed form and the design space can be of high dimensionality. The approximate coordinate exchange algorithm is proposed for this maximisation problem where a Gaussian process emulator is used to approximate the objective function. The algorithm can be used for arbitrary utility functions meaning we can consider fully Bayesian optimal design. It can also be used for those utility functions that result in pseudo-Bayesian designs such as the popular Bayesian D-optimality. The algorithm is demonstrated on a range of examples

Multivariate emulation of computer simulators: model selection and diagnostics with application to a humanitarian relief model

We present a common framework for Bayesian emulation methodologies for multivariate-output simulators, or computer models, that employ either parametric linear models or nonparametric Gaussian processes. Novel diagnostics suitable for multivariate covariance-separable emulators are developed and techniques to improve the adequacy of an emulator are discussed and implemented. A variety of emulators are compared for a humanitarian relief simulator, modelling aid missions to Sicily after a volcanic eruption and earthquake, and a sensitivity analysis is conducted to determine the sensitivity of the simulator output to changes in the input variables. The results from parametric and nonparametric emulators are compared in terms of prediction accuracy, uncertainty quantification and scientific interpretability

Bayesian Optimal Design for Ordinary Differential Equation Models

Bayesian optimal design is considered for experiments where it is hypothesised that the responses are described by the intractable solution to a system of non-linear ordinary differential equations (ODEs). Bayesian optimal design is based on the minimisation of an expected loss function where the expectation is with respect to all unknown quantities (responses and parameters). This expectation is typically intractable even for simple models before even considering the intractability of the ODE solution. New methodology is developed for this problem that involves minimising a smoothed stochastic approximation to the expected loss and using a state-of-the-art stochastic solution to the ODEs, by treating the ODE solution as an unknown quantity. The methodology is demonstrated on three illustrative examples and a real application involving estimating the properties of human placentas

Zero-Truncated Modelling Meta-Analysis for When Studies with No Events Are Systematically Excluded: Estimating Completed Suicide After Bariatric Surgery

Meta-analysis is a well-established method for integrating results from several independent studies to estimate a common quantity of interest. However, meta-analysis is prone to selection bias, notably when particular studies are systematically excluded. This can lead to bias in estimating the quantity of interest. Motivated by a meta-analysis to estimate the rate of completed suicide after bariatric surgery, where studies which reported no suicides were excluded, a novel zero-truncated count modelling approach was developed. This approach addresses heterogeneity, both observed and unobserved, through covariate and overdispersion modelling, respectively. Additionally, through the Horvitz-Thompson estimator, an approach was developed to estimate the number of excluded studies, a quantity of potential interest for researchers. Uncertainty quantification for both estimation of suicide rates and number of excluded studies was achieved through a parametric bootstrapping approach.Comment: 15 page

acebayes: An R Package for Bayesian Optimal Design of Experiments via Approximate Coordinate Exchange

We describe the R package acebayes and demonstrate its use to find Bayesian optimal experimental designs. A decision-theoretic approach is adopted, with the optimal design maximizing an expected utility. Finding Bayesian optimal designs for realistic problems is challenging, as the expected utility is typically intractable and the design space may be high-dimensional. The package implements the approximate coordinate exchange algorithm to optimize (an approximation to) the expected utility via a sequence of conditional one-dimensional optimization steps. At each step, a Gaussian process regression model is used to approximate, and subsequently optimize, the expected utility as the function of a single design coordinate (the value taken by one controllable variable for one run of the experiment). In addition to functions for bespoke design problems with user-defined utility functions, acebayes provides functions tailored to finding designs for common generalized linear and nonlinear models. The package provides a step-change in the complexity of problems that can be addressed, enabling designs to be found for much larger numbers of variables and runs than previously possible. We provide tutorials on the application of the methodology for four illustrative examples of varying complexity where designs are found for the goals of parameter estimation, model selection and prediction. These examples demonstrate previously unseen functionality of acebayes

Bayesian Optimal Design for Ordinary Differential Equation Models

Bayesian optimal design is considered for experiments where it is hypothesised that the responses are described by the intractable solution to a system of non-linear ordinary differential equations (ODEs). Bayesian optimal design is based on the minimisation of an expected loss function where the expectation is with respect to all unknown quantities (responses and parameters). This expectation is typically intractable even for simple models before even considering the intractability of the ODE solution. New methodology is developed for this problem that involves minimising a smoothed stochastic approximation to the expected loss and using a state-of-the-art stochastic solution to the ODEs, by treating the ODE solution as an unknown quantity. The methodology is demonstrated on three illustrative examples and a real application involving estimating the properties of human placentas

Gibbs optimal design of experiments

Bayesian optimal design of experiments is a well-established approach to planning experiments. Briefly, a probability distribution, known as a statistical model, for the responses is assumed which is dependent on a vector of unknown parameters. A utility function is then specified which gives the gain in information for estimating the true value of the parameters using the Bayesian posterior distribution. A Bayesian optimal design is given by maximising the expectation of the utility with respect to the joint distribution given by the statistical model and prior distribution for the true parameter values. The approach takes account of the experimental aim via specification of the utility and of all assumed sources of uncertainty via the expected utility. However, it is predicated on the specification of the statistical model. Recently, a new type of statistical inference, known as Gibbs (or General Bayesian) inference, has been advanced. This is Bayesian-like, in that uncertainty on unknown quantities is represented by a posterior distribution, but does not necessarily rely on specification of a statistical model. Thus the resulting inference should be less sensitive to misspecification of the statistical model. The purpose of this paper is to propose Gibbs optimal design: a framework for optimal design of experiments for Gibbs inference. The concept behind the framework is introduced along with a computational approach to find Gibbs optimal designs in practice. The framework is demonstrated on exemplars including linear models, and experiments with count and time-to-event responses